Hierarchical Regularization of Building Boundaries in Noisy Aerial Laser Scanning and Photogrammetric Point Clouds

Aerial laser scanning or photogrammetric point clouds are often noisy at building boundaries. In order to produce regularized polygons from such noisy point clouds, this study proposes a hierarchical regularization method for the boundary points. Beginning with detected planar structures from raw point clouds, two stages of regularization are employed. In the first stage, the boundary points of an individual plane are consolidated locally by shifting them along their refined normal vector to resist noise, and then grouped into piecewise smooth segments. In the second stage, global regularities among different segments from different planes are softly enforced through a labeling process, in which the same label represents parallel or orthogonal segments. This is formulated as a Markov random field and solved efficiently via graph cut. The performance of the proposed method is evaluated for extracting 2D footprints and 3D polygons of buildings in metropolitan area. The results reveal that the proposed method is superior to the state-of-art methods both qualitatively and quantitatively in compactness. The simplified polygons could fit the original boundary points with an average residuals of 0.2 m, and in the meantime reduce up to 90% complexities of the edges. The satisfactory performances of the proposed method show a promising potential for 3D reconstruction of polygonal models from noisy point clouds

Optimal Self-Calibration Strategies in the Combined Bundle Adjustment of Aerial–Terrestrial Integrated Images

Accurate combined bundle adjustment (BA) is a fundamental step for the integration of aerial and terrestrial images captured from complementary platforms. In traditional photogrammetry pipelines, self-calibrated bundle adjustment (SCBA) improves the BA accuracy by simultaneously refining the interior orientation parameters (IOPs), including lens distortion parameters, and the exterior orientation parameters (EOPs). Aerial and terrestrial images separately processed through SCBA need to be fused using BA. Thus, the IOPs in the aerial–terrestrial BA must be properly treated. On one hand, the IOPs in one flight should be identical for the same images in physics. On the other hand, the IOP adjustment in the cross-platform-combined BA may mathematically improve the aerial–terrestrial image co-registration degree in 3D space. In this paper, the impacts of self-calibration strategies in combined BA of aerial and terrestrial image blocks on the co-registration accuracy were investigated. To answer this question, aerial and terrestrial images captured from seven study areas were tested under four aerial–terrestrial BA scenarios: the IOPs for both aerial and terrestrial images were fixed; the IOPs for only aerial images were fixed; the IOPs for only terrestrial images were fixed; the IOPs for both images were adjusted. The cross-platform co-registration accuracy for the BA was evaluated according to independent checkpoints that were visible on the two platforms. The experimental results revealed that the recovered IOPs of aerial images should be fixed during the BA. However, when the tie points of the terrestrial images are comprehensively distributed in the image space and the aerial image networks are sufficiently stable, refining the IOPs of the terrestrial cameras during the BA may improve the co-registration accuracy. Otherwise, fixing the IOPs is the best solution

Unsupervised stepwise extraction of offshore aquaculture ponds using super-resolution hyperspectral images

Despite offshore aquaculture brings great economic benefits, it also has destructive effects on the ecological environment of coastal regions. Therefore, the accurate monitoring of offshore aquaculture areas is vital. Existing methods for extracting aquaculture ponds are still limited by the lack of high-resolution hyperspectral imagery and inadequate feature extraction. In this paper, we proposed an unsupervised aquaculture ponds extraction method based on hyperspectral imagery super-resolution, feature fusion and stepwise extraction strategy. First, the resolution of the original hyperspectral imagery is enhanced via a deep learning-based super-resolution method. Then we introduce a feature fusion method by combining multi-dimensional features including elevation, reflectance, biochemistry, principal component analysis, and fast-Fourier transform features to enhance the feature sensitivity to the aquaculture ponds. The aquaculture ponds are finally extracted via a step-wise extraction strategy. Experiments on ZH-1 and GF-7 imageries are conducted, and the experimental results show that our method achieves an OA of 97.9% on the aquaculture pond extraction and that the method can be generalized for the object extraction on multispectral datasets. Ablation experiments show that all three of our innovative modules can effectively improve the extraction accuracy and confirm the generalization of the method

Optimal Self-Calibration Strategies in the Combined Bundle Adjustment of Aerial–Terrestrial Integrated Images

Accurate combined bundle adjustment (BA) is a fundamental step for the integration of aerial and terrestrial images captured from complementary platforms. In traditional photogrammetry pipelines, self-calibrated bundle adjustment (SCBA) improves the BA accuracy by simultaneously refining the interior orientation parameters (IOPs), including lens distortion parameters, and the exterior orientation parameters (EOPs). Aerial and terrestrial images separately processed through SCBA need to be fused using BA. Thus, the IOPs in the aerial–terrestrial BA must be properly treated. On one hand, the IOPs in one flight should be identical for the same images in physics. On the other hand, the IOP adjustment in the cross-platform-combined BA may mathematically improve the aerial–terrestrial image co-registration degree in 3D space. In this paper, the impacts of self-calibration strategies in combined BA of aerial and terrestrial image blocks on the co-registration accuracy were investigated. To answer this question, aerial and terrestrial images captured from seven study areas were tested under four aerial–terrestrial BA scenarios: the IOPs for both aerial and terrestrial images were fixed; the IOPs for only aerial images were fixed; the IOPs for only terrestrial images were fixed; the IOPs for both images were adjusted. The cross-platform co-registration accuracy for the BA was evaluated according to independent checkpoints that were visible on the two platforms. The experimental results revealed that the recovered IOPs of aerial images should be fixed during the BA. However, when the tie points of the terrestrial images are comprehensively distributed in the image space and the aerial image networks are sufficiently stable, refining the IOPs of the terrestrial cameras during the BA may improve the co-registration accuracy. Otherwise, fixing the IOPs is the best solution

Urban Building Extraction and Modeling Using GF-7 DLC and MUX Images

Urban modeling and visualization are highly useful in the development of smart cities. Buildings are the most prominent features in the urban environment, and are necessary for urban decision support; thus, buildings should be modeled effectively and efficiently in three dimensions (3D). In this study, with the help of Gaofen-7 (GF-7) high-resolution stereo mapping satellite double-line camera (DLC) images and multispectral (MUX) images, the boundary of a building is segmented via a multilevel features fusion network (MFFN). A digital surface model (DSM) is generated to obtain the elevation of buildings. The building vector with height information is processed using a 3D modeling tool to create a white building model. The building model, DSM, and multispectral fused image are then imported into the Unreal Engine 4 (UE4) to complete the urban scene level, vividly rendered with environmental effects for urban visualization. The results of this study show that high accuracy of 95.29% is achieved in building extraction using our proposed method. Based on the extracted building vector and elevation information from the DSM, building 3D models can be efficiently created in Level of Details 1 (LOD1). Finally, the urban scene is produced for realistic 3D visualization. This study shows that high-resolution stereo mapping satellite images are useful in 3D modeling for urban buildings and can support the generation and visualization of urban scenes in a large area for different applications